Spectroscopic imaging apparatus and fluorescence observation apparatus

ABSTRACT

A spectroscopic imaging apparatus according to an embodiment of the present technology includes a spectroscopic section, an image sensor, and a control unit. The spectroscopic section disperses incident light for each wavelength. The image sensor is configured to be capable of setting an exposure time or a gain in a unit of a pixel, and detects light of each wavelength dispersed in the spectroscopic section. The control unit is configured to be capable of setting the exposure time or the gain of the image sensor in a unit of a predetermined pixel area.

TECHNICAL FIELD

The present technology relates to, for example, a spectroscopic imagingapparatus and a fluorescence observation apparatus used for diagnosis ofa pathological image.

BACKGROUND ART

A pathological image diagnosis using fluorescence staining has beenproposed as a highly quantitative and polychromatic approach (see PatentLiterature 1, for example). As compared with colored staining, afluorescent approach has advantages in that multiplexing is easilyperformed and detailed diagnostic information is obtained. In afluorescence imaging other than the pathological diagnosis, an increasein the number of colors makes it possible to examine various antigensexpressed in a sample at a time.

CITATION LIST Patent Literature

Patent Literature 1: Japanese Patent No. 4,452,850

DISCLOSURE OF INVENTION Technical Problem

A spectroscopic observation apparatus which expands a horizontal axis ofan area sensor to space and a vertical axis thereof to wavelength caneasily obtain a spectroscopic spectrum of one line on the sample.However, in a case where a bright wavelength band, a very darkwavelength band, or the like is mixed in the spectrum, a dynamic rangeof the sensor itself is insufficient, a dark portion collapses, or abright portion saturates, which results that sufficient data is notobtained. On the other hand, in order to solve this problem, if a sensorhaving a large recording capacity is used, a storage capacity increasesin a target such as the pathological image in which the number of wholepixels becomes enormous, and new problems arise such as a decrease inaccessibility to data and a slow operation of an entire system.

In view of the above circumstances, an object of the present technologyis to provide a spectroscopic imaging apparatus and a fluorescenceobservation apparatus capable of recording in a high dynamic range whilesuppressing a recording capacity of a sensor.

Solution to Problem

A spectroscopic imaging apparatus according to an embodiment of thepresent technology includes a spectroscopic section, an image sensor,and a control unit.

The spectroscopic section disperses incident light for each wavelength.

The image sensor is configured to be capable of setting an exposure timeor a gain in a unit of a pixel, and detects light of each wavelengthdispersed in the spectroscopic section.

The control unit is configured to be capable of setting the exposuretime or the gain of the image sensor in a unit of a predetermined pixelarea.

According to the above-described spectroscopic imaging apparatus, it ispossible to obtain optimum exposure conditions and to expand a dynamicrange of a spectrum to be recorded.

The spectroscopic section may be configured to disperse the incidentlight in one axial direction for each wavelength, and the control unitmay be configured to set the exposure time of the image sensor in a unitof line perpendicular to the one axial direction.

The image sensor may further include a pixel section and a calculationsection that calculates a pixel value from image data output from thepixel section. In this case, the control unit is configured to set thegain used for calculating the pixel value in the unit of thepredetermined pixel area.

The control unit may include an evaluation section that obtains anemission spectrum of the incident light on the basis of an output of theimage sensor, and a storage section that stores a plurality of referencecomponent spectra and an autofluorescence spectrum. The evaluationsection is configured to calculate a component ratio of the emissionspectrum such that a linear sum of a plurality of the referencecomponent spectra and the autoluminescence spectrum is obtained.

The evaluation section is configured to calibrate at least one of theemission spectrum or the component spectra on the basis of the exposuretime or the gain set for each predetermined pixel area.

The evaluation section may be configured to determine whether or notthere is a pixel whose pixel value reaches saturation from a capturedspectrum, and exclude the pixel reaching the saturation from calculationof a component ratio of the captured spectrum.

A fluorescence observation apparatus according to an embodiment of thepresent technology includes a stage, an excitation section, aspectroscopic section, an image sensor, and a control unit.

The stage is configured to be capable of supporting a fluorescencestained pathological specimen.

The excitation section irradiates the pathological specimen on the stagewith line illumination.

The spectroscopic section disperses the fluorescence excited by the lineillumination for each wavelength.

The image sensor is configured to be capable of setting an exposure timeor a gain in a unit of a pixel, and detects light of each wavelengthdispersed in the spectroscopic section.

The control unit is configured to set the exposure time or the gain ofthe image sensor in a unit of a predetermined pixel area.

The fluorescence observation apparatus may further include a displaysection for displaying the fluorescence spectrum on the basis of anoutput of the image sensor.

The display section may have an operation area for receiving an input ofan exposure time or a gain in the unit of the predetermined pixel area.

The display section may have a display area for displaying a spectrumand a histogram after setting on the basis of the exposure time or thegain set.

Advantageous Effects of Invention

As described above, according to the present technology, it is possibleto perform recording in a high dynamic range while suppressing arecording capacity of the sensor.

Note that the effects described here are not necessarily limitative, andany of the effects described in the present disclosure may be provided.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram showing a basic configuration of aspectroscopic imaging apparatus according to an embodiment of thepresent technology.

FIG. 2 is a schematic diagram showing an optical system of afluorescence observation apparatus provided with the spectroscopicimaging apparatus.

FIG. 3 a schematic diagram of a pathological specimen of an observationtarget.

FIG. 4 is a block diagram showing a configuration of the fluorescenceobservation apparatus.

FIG. 5 is a block diagram showing a configuration of a detection sectionand its periphery in the fluorescence observation apparatus.

FIG. 6 is a schematic diagram for explaining a relationship between apixel section and an emission spectrum.

FIG. 7 is an explanatory diagram showing a relationship between theemission spectrum and a dynamic range in a detection area.

FIG. 8 is a flowchart showing a processing procedure up to componentseparation calculation of the emission spectrum executed in a controlunit.

FIG. 9 is a flowchart showing an example of a saturation processingprocedure in the embodiment.

FIG. 10 is a schematic diagram explaining an example of the saturationprocessing.

FIG. 11 is a schematic diagram of a display section in the fluorescenceobservation apparatus.

FIG. 12 is a diagram showing an example of a screen configuration of asetting area of an excitation section in the display section.

FIG. 13 is a diagram showing an example of a screen configuration of adetection setting area of a fluorescence spectrum from one lineillumination in the display section.

FIG. 14 is a diagram showing an example of a screen configuration of adetection setting area of a fluorescence spectrum from other lineillumination in the display section.

FIG. 15 is a diagram for explaining a histogram window in the displaysection.

FIG. 16 is a block diagram of the fluorescence observation apparatus forexplaining processing performed in the control unit.

FIG. 17 is a schematic block diagram showing one modification of thefluorescence observation apparatus.

FIG. 18 is a schematic block diagram showing other modification of thefluorescence observation apparatus.

MODE(S) FOR CARRYING OUT THE INVENTION

Embodiments according to the present technology will now be describedbelow with reference to the drawings.

Outline of Apparatus

FIG. 1 is a schematic diagram showing a basic configuration of aspectroscopic imaging apparatus 10 according to an embodiment of thepresent technology.

As shown in the same figure, the spectroscopic imaging apparatus 10 is aline scan type imaging spectrometer, and includes a spectroscopicsection 11 and a detection section 12. The spectroscopic section 11 hasa slit 111 parallel with the X-axis direction, and a wavelengthdispersive element 112. The detection section 12 includes an imagesensor (area sensor) 121 including a solid-state imaging element such asa CMOS (Complementary Metal-Oxide Semiconductor) and a CCD(Charge-Coupled Device).

The slit 111 extracts a spatial component in the X-axis direction ofincident light (fluorescence) from a sample (not shown) on the xy plane.The wavelength dispersive element 112 disperses incident light Lxpassing through the slit 111 for each wavelength to image the imagesensor 121. As the wavelength dispersive element 112, a prism or adiffraction grating is typically used to separate each wavelength bandof the incident light Lx in the Y-axis direction. The image sensor 121obtains a spectral image of (X, A) of the incident light L1wavelength-separated in the wavelength dispersive element 112. Byincorporating a mechanism for scanning the sample in the Y-axisdirection, a spectral image of (X, Y, A) can be obtained.

The image sensor 121 is configured to be capable of setting an exposuretime or a gain for a unit of a pixel, as will be described later. Byadjusting the exposure time or the gain depending on a light receivingarea of the light in each wavelength band, it is possible to suppresssaturation for the light in a bright wavelength band and to obtain aspectral image with sufficient sensitivity for the light in a darkwavelength band.

Furthermore, the image sensor 121 is configured to read only a part ofan area from a read-out area of in a full frame. As a result, a framerate can be improved by an amount corresponding to a reduction of theread-out area. Furthermore, it is possible to divide any area of theread-out area into plural, and set a different gain and exposure time ineach area.

Fluorescence Observation Apparatus

FIG. 2 is a schematic diagram showing an optical system of thefluorescence observation apparatus 100 including the spectroscopicimaging apparatus 10 of the present embodiment.

As shown in the same figure, the fluorescence observation apparatus 100includes a spectroscopic section 11, a detection section 12, and afluorescence excitation section 13. The fluorescent excitation section13 includes an excitation light optical system 131, a filter block 132,and an objective lens 133.

The excitation light optic system 131 includes one light source or aplurality of light sources capable of emitting excitation light. As thelight source, a light emitting diode (LED), a laser diode (LD), amercury lamp, or the like is used. The excitation light isline-illuminated and irradiates the sample S on a stage 20 in parallelwith the xy plane.

The sample S is typically formed of a slide including an observationtarget Sa such as a tissue section shown in FIG. 3. However, it shouldbe appreciated that the sample S may be formed of something other thansuch a slide. The sample S (observation target Sa) is stained by aplurality of fluorescent pigments excited by irradiation of theexcitation light.

A filter block 132 includes a dichroic mirror, a band-pass filter, andthe like. The dichroic mirror reflects the excitation light from theexcitation light optical system 131 toward the objective lens 133, andtransmits the fluorescence from the sample S transmitted through theobjective lens 133 toward the spectroscopic section 11. The bandpassfilter has a band-pass characteristic of cutting the wavelength band ofthe excitation light of the light directing from the sample S toward thespectroscopic section 11.

FIG. 4 is a block diagram showing a configuration of the fluorescenceobservation apparatus 100. The fluorescence observation apparatus 100includes an apparatus main body 1, a control unit 2, and a displaysection 3.

The apparatus main body 1 includes the stage 20, an excitation lightsource (excitation section) 101, a spectroscopic imaging section 102, anobservation optical system 103, a scanning mechanism 104, a focusmechanism 105, a non-fluorescence observation section 106, and the like.

The excitation light source 101 corresponds to the excitation opticalsystem 131, and the spectroscopic imaging section 102 corresponds to thespectroscopic section 11 and the detection section 12. The observationoptical system 103 corresponds to the filter block 132 and the objectivelens 133.

The scanning mechanism 104 is typically formed of an XY moving mechanismthat moves in parallel with the stage 20 in at least two directions ofthe X and Y axes. In this case, an image-capturing area Rs is dividedinto a plurality of areas in the X-axis direction, for example, asillustrated in FIG. 3, and an operation is repeatedly performed, i.e.,scanning the sample S in the Y-axis direction, subsequently moving thesample S in the X-axis direction, and further performing scanning in theY-axis direction. As a result, a large-area spectral image can beobtained, and, for example, in the case of a pathological slide or thelike, WSI (Whole slide imaging) can be obtained.

The focus mechanism 105 moves the stage 20 or the objective lens 133 toan optimal focal position in a direction perpendicular to the X-axis andthe Y-axis. The non-fluorescence observation section 106 is used fordark field observation, bright field observation, or the like of thesample S, but may be omitted as necessary.

The fluorescence observation apparatus 100 may be connected to a controlsection 80 for controlling a fluorescence excitation section (control ofLD or shutter), an XY stage as the scanning mechanism, a spectroscopicimaging section (camera), a focusing mechanism (detection section andZ-stage), a non-fluorescence observation section (camera), and the like.

Image Sensor

FIG. 5 is a block diagram showing a configuration of the detectionsection 12 and its periphery.

As shown in the same figure, the detection section 12 includes an imagesensor 121, and a signal processing circuit 122. The image sensor 121includes a pixel section 30 and a calculation section 31.

The pixel section 30 outputs charge information corresponding to theexposure time by photoelectric conversion in each pixel of a pixel arrayof the Bayer array consisting of RGB pixels, for example. The pixelsection 30 is set to a different exposure time in a unit of a pixel area(e.g., row (line) unit) by a control of the control unit 2 (shuttercontrol). From the row to be subjected to long-time exposure,high-sensitivity pixel information 311 corresponding to accumulatedcharge based on the long-time exposure is output. From the row to besubjected to short-time exposure, low-sensitivity pixel information 312corresponding to the accumulated charge based on the short-time exposureis output.

The calculation section 31 calculates a pixel value from image dataoutput from the pixel section 30. In the present embodiment, thecalculation section 31 inputs the high-sensitivity pixel information 311and the low-sensitivity pixel information 312 output from the pixelsection 30, and has an image information synthesizing section 313 forgenerating one piece of image information on the basis of the inputinformation. The output of the image information synthesizing section313 is input to the signal processing circuit 122. The signal processingcircuit 122 performs signal processing, for example, such as whitebalance (WB) adjustment and y correction to generate an output image.The output image is supplied to the control unit 2, stored in a storagesection 21 described later, or output to the display section 3.

The image sensor 121 obtains fluorescence spectroscopic data (x, X)utilizing the pixel array in the Y-axis direction (vertical direction)of the pixel section 30 as a channel of the wavelength. The obtainedspectroscopic data (x, X) is recorded in the control unit 2 (storagesection 21) in a state in which whether the spectroscopic data excitedfrom which excitation wavelength is tied.

An exposure time of the pixel section 30 is set for each predeterminedpixel area by the control unit 2. In the present embodiment, since thewavelength dispersive element 112 in the spectroscopic section 11wavelength-separates the incident light Lx (see FIG. 1) in the Y-axisdirection, light having a different wavelength in the Y-axis direction(emission spectrum) reaches the pixel section 30 of the image sensor121. Therefore, in the present embodiment, as described above, by thecontrol of the control unit 2 (shutter control), the exposure time ofthe pixel section 30 is set in a unit of a line in parallel with theX-axis direction perpendicular to the Y-axis direction.

The control unit 2 is further configured to be capable of individuallysetting the gain for sensitivity compensation multiplied by each of thehigh-sensitivity pixel information 311 and the low-sensitivity pixelinformation 312 in the image information synthesizing section 313 of thecalculation section 31 in the unit of the pixel area. Thus, it becomespossible to increase the sensitivity of the low-sensitivity pixelinformation while suppressing the saturation of the high-sensitivitypixel information 311.

Set values of the exposure time and the gain are not particularlylimited, and may be arbitrary values or values based on an emissionspectrum intensity of the pigment measured in advance. For example, whenthe exposure time of a low-sensitivity pixel area and the gain of thepixel value are set to 1, the exposure time of a high-sensitivity pixelinformation area and the gain of the pixel value are set to, forexample, a range of about 1.5 to 5.0.

In addition, it is not limited to the case where both the exposure timeand the gain are set, and it may be set so that only the exposure timeis adjustable, or it may be set so that only the gain is adjustable.Alternatively, one of the exposure time and the gain as a main setvalue, and the other may be a supplemental set value. For example, bysetting the exposure time as the main set value, the image data withgood S/N can be obtained.

FIG. 6 is a schematic diagram for explaining a relationship between thepixel section 30 and the emission spectrum.

As shown in the same figure, the control unit 2 determines a detectionarea from a wavelength range of the emission spectrum, from atransmission wavelength range of the filter block 132 (see FIG. 2), andfrom an entire read-out area of the image sensor 121 (pixel section 30).In the case of fluorescence imaging, the filter block 132 generally hasa bandpass characteristic for cutting off excitation light. Therefore,if a plurality of the excitation wavelengths exists, a band (opaque bandDZ) in which the wavelengths are not transmitted as shown in the samefigure is generated. The control unit 2 excludes an area that does notinclude such a signal to be detected from the detection area.

As shown in FIG. 6, when areas located above and below an impermeablezone DZ are taken as ROI1 and ROI2, respectively, the emission spectraof the pigments having the corresponding peaks (hereinafter, alsoreferred to as fluorescent spectrum) is detected. FIG. 7 is anexplanatory diagram showing the relationship between the emissionspectrum and the dynamic range in the detection area, (a) of the samefigure shows obtained data before setting the exposure time and the gain(exposure time or gain is same in each detection area), and (b) of thesame figure shows obtained data after setting the exposure time and thegain, respectively.

As shown in FIG. 7(a), the pigment of ROI1 has a strong spectralintensity and saturates beyond the dynamic range of detection, whereasthe pigment of ROI2 has a weak intensity. In the present embodiment, asshown in FIG. 7(b), the exposure time of (X, A) area corresponding tothe ROI1 is set to be relatively short (or gain is set to be relativelysmall), and conversely, the exposure time of (X, A) area correspondingto the ROI2 is set to be relatively long (or gain is set to berelatively large). As a result, both dark and light pigments can becaptured with suitable exposure. Coordinate information of the detectedarea, such as the ROI1 and the ROI2, and the gain, and information aboutthe exposure duration are stored in the storage section 21 of thecontrol unit 2.

Control Unit

The fluorescence spectrum obtained by the detection section 12 includingthe image sensor 121 (spectroscopic imaging section 102) is output tothe control unit 2. Captured data of a multiple fluorescence spectrumcan be quantitatively evaluated by component analysis (color separation)on the basis of the spectrum of the pigment alone or the like. Thecontrol unit 2, as shown in FIG. 4, includes the storage section 21 andthe evaluation section 22.

The control unit 2 may be implemented by hardware elements used in acomputer, such as a CPU (Central Processing Unit), a RAM (Random AccessMemory), and a ROM (Read Only Memory), and by necessary software.Instead of or in addition to the CPU, a PLD (Programmable Logic Device)such as an FPGA (Field Programmable Gate Array), or a DSP (DigitalSignal Processor), other ASIC (Application Specific Integrated Circuit(ASIC), or the like may be used.

The storage section 21 stores in advance a component spectrum serving asa plurality of references of the pigment alone for staining the sample Sand an autoluminescence spectrum of the sample S (hereinafter, alsocollectively referred to as standard spectra). An evaluation section 22separates the emission spectrum of the sample S obtained by the imagesensor 121 into a spectrum derived from the pigment and theautoluminescence spectrum on the basis of the standard spectra stored inthe storage section 21, and calculates each component ratio. In thepresent embodiment, the component ratio of the emission spectrum of thecaptured sample S is calculated so as to be a linear sum of standardspectra.

On the other hand, the emission spectrum of the sample S obtained by theimage sensor 121 is modulated from the original spectrum because theexposure time and the gain are individually set for each detection area.Therefore, if the data obtained by the image sensor 121 is used as itis, a color separation calculation of the component spectrum may not beperformed accurately.

Accordingly, the evaluation section 22 is configured to calibrate atleast one of the emission spectrum or the reference component spectrumon the basis of the exposure time and the gain set for eachpredetermined pixel area (detection area) of the image sensor 121.

FIG. 8 is a flowchart showing a processing procedure up to the componentseparation calculation of the emission spectrum executed in the controlunit 2. Hereinafter, the emission spectrum of the sample S obtained bythe image sensor 121 is also referred to as the captured spectrum.

As shown in the same figure, the control unit 2 sets the exposure timeand the gain of the detection area of the pixel section 30 of the imagesensor 121 (Step 101). These set values are input by the user via thedisplay section 3 to be described later. After the exposure time and thegain set are recorded to the storage section 21, the control unit 2obtains the captured spectrum of the sample S via the image sensor 121(Steps 102 and 103).

The control unit 2 demodulates the captured spectrum on the basis of theset gain and the set exposure time of each detection area, or calibratesthe captured spectrum by modulating the standard spectra stored in thestorage section 21 (Step 104). In other words, on the basis of the setexposure time and the set gain, the captured spectrum and the standardspectra are converted into a common intensity axis. The intensity axisinclude, for example, a charge number per unit time [e⁻], spectrumradiance [W/(sr·m²·nm)], or the like. In a case where the standardspectra are changed, the standard spectra are multiplied by a relativeintensity ratio of each detection area at the time of capturing.Thereafter, saturation processing (Step 105), which will be describedlater, is performed as necessary, and then component separationcalculation of the captured spectrum is performed (Step 106).

On the other hand, in a case where a spectrum capturing of a multiplefluorescent sample is performed by spectroscopy, it is important to seta parameter such as the exposure time and the gain of each pixel so thatcapturing can be done without saturation in advance. However, in an WSIor the like, it is very difficult to obtain an optimal exposure in allareas of the sample, and a time loss is large as well. When saturationoccurs during capturing, a peak of the spectrum reaches the limit at anAD (Analog to Digital) max value of the sensor, making it impossible tocapture a correct spectrum. Therefore, there arises a problem that adeviation from the component spectrum (standard spectra) prepared inadvance for the color separation calculation becomes large, and acorrect calculation cannot be performed.

Therefore, in the present embodiment, the saturation processingdescribed later is performed in addition to an expansion of the dynamicrange by a ROI (Region of interest) setting. This makes it possible tocorrectly perform the color separation calculation even when there issome saturation in the captured spectrum, thereby reducing the number ofretries of capturing.

The saturation processing in the present embodiment executes processingof specifying the pixel in which the saturation occurs and excluding thepixel from the calculation. An example of the processing procedure isshown in FIG. 9.

FIG. 9 is a flow chart showing the saturation processing procedure.

As shown in the same figure, the control unit 2 executes processing ofgenerating a saturation detection array from the obtained capturedspectrum (Step 201). As shown in FIG. 10, the presence or absence ofsaturation of the captured spectrum is determined for each wavelength(channel), and the saturation detection array is generated in which achannel without saturation is set to “1” and a channel with saturationis set to “0”.

The presence or absence of saturation is determined by referring to thepixel value of each detection area and whether or not it reaches amaximum luminance value. Since the pixel area reaching the maximumluminance value is estimated to be saturated in comparison with theoriginal correct spectrum, the channel of the reference spectrumcorresponding to the pixel area (channel) is removed from the componentseparation calculation.

In general, the number of channels (CH number) of the wavelengthrecorded by the spectrum capturing is often larger than the number ofcomponents to be finally output. Therefore, if the number of effectivechannels in which no saturation occurs is larger than the number ofcomponents, even if the data of the channel in which saturation occursis removed from the calculation, the component separation calculationcan be performed.

When the number of effective channels (number of channels determined as“1”) in the generated array is larger than the number of components(number of channels) to be finally output, processing of multiplying thesaturation detection array by the captured spectrum and the referencespectrum is executed (Steps 203 and 204). Otherwise, the calculation isimpossible, and therefore, the processing is ended without executing thecomponent separation calculation. As a result, since the channel inwhich the saturation occurs is excluded from a calculation of a leastsquares method, it is possible to perform a component ratio calculationonly with the correctly measured wavelength.

According to the present embodiment as described above, with respect tothe image sensor 121 capable of changing a gain setting and the exposuretime of any detection area, a spectroscopic imaging optical system isprovided to expand the horizontal axis of the image sensor 121 to spaceand the vertical axis thereof to wavelength. From each area of the imagesensor 121, by setting so as to read only the detection area, furtherdividing the detection area into two or more two-dimensional spaces ROIof wavelength x space, and by setting a combination of different gainsand exposure times to each detection area, the optimum exposurecondition is obtained and it is also possible to expand the dynamicrange of the spectrum to be recorded.

For example, when a multiple fluorescent stained sample is taken, a bluefluorescent pigment may have a very high intensity compared to a redfluorescent pigment. Under such conditions, the exposure time of theblue wavelength band is shortened, the gain is set to be lower, theexposure time of the red wavelength band is lengthened, and the gain isset to be higher. As a result, recording with a shallow bit range can beperformed, so that recording with a high dynamic range can be performedwhile suppressing the recording capacity of the sensor.

The detection area of the image sensor 121 is set from the spectrum ofthe object to be measured within the sensitivity area of the sensor. Ifthere are a non-transparent band such as a notch filter, and an areawhere light is not present in an observation light path, a recordingframe rate can be improved by excluding them from the read-out area.

Furthermore, according to the present embodiment, when a color mixingratio of each pigment is separately calculated from the obtainedspectrum, even if there is some saturation in the captured spectrum,color separation by spectrum fitting can be performed by generating asaturation detection array (see FIG. 10) for distinguishing a saturatedwavelength from the other wavelengths.

Display Section

A problem of a capturing parameter setting by the ROI is that acapturing condition is difficult to be understood for a user. Becausethe data is three-dimensional of a space and a wavelength, it is hard tosee where the saturation occurs and a signal of which wavelength isinsufficient. A section that performs ROI setting and displaying needsto be capable of comprehensively displaying and setting a relationshipbetween a setting parameter and a capture range, a relationship betweenthe setting parameter and a sensor output, and the like.

Therefore, in the present embodiment, the display section 3 isconfigured as follows and details of the display section 3 will bedescribed below. Here, as an example, a configuration of the displaysection 3 assuming a multiple fluorescence imaging will be described.

FIG. 11 is a schematic diagram explaining the display section 3. Thedisplay section 3 is configured to be capable of displaying thefluorescence spectrum of the sample S on the basis of the output of theimage sensor 121. The display section 3 may be constituted by a monitormounted integrally to the control unit 2 or may be a display apparatusconnected to the control unit 2. The display section 3 includes adisplay element such as a liquid crystal device or an organic EL device,and a touch sensor, and is configured as a UI (User Interface) thatdisplays a setting for inputting a capturing condition, a capturedimage, and the like.

As shown in FIG. 11, the display section 3 includes a main screen 301, athumbnail image display screen 302, a slide information display screen303, and a captured slide list display screen 304. The main screen 301includes a display area 305 of a control button (key) for capturing, asetting area 306 of an excitation laser (line illumination), detectionsetting areas 307 and 308 of the spectrum, a spectrum automatic settingcontrol area 309, and the like. There may be at least one of these areas305 to 309, and other display area may be included in one display area.

The fluorescence observation apparatus 100 sequentially performs atakeout of a slide (sample S) from a slide rack (not shown), reading ofslide information, capturing of a thumbnail of the slide, setting of anexposure time, and the like. The slide information includes patientinformation, a tissue site, a disease, staining information, and thelike, and is read from a bar code, a QR code (registered trademark), orthe like attached to the slide. The thumbnail image and the slideinformation of the sample S are respectively displayed on the displayscreens 302 and 303. The captured slide information is displayed on thescreen 304 as a list.

In addition to the fluorescence image of the sample S, a capturing stateof the slide currently captured is displayed on the main screen 301. Theexcitation laser is displayed or set in the setting area 306, and thefluorescence spectrum derived from the excitation laser is displayed orset in the detection setting areas 307 and 308.

FIG. 12 is a diagram showing an example of a screen configuration of thesetting area 306 of the excitation laser. Here, ON/OFF of outputs ofrespective excitation lines L1-L4 are selected or switched by a touchoperation to each checkbox 81. Furthermore, a magnitude of the output ofeach light source is set through an operation section 82.

FIG. 13 shows an example of a screen configuration of a spectrumdetection setting area 307 in an excitation line 1, and FIG. 14 shows anexample of a screen configuration of the spectrum detection setting area308 in an excitation line 2. In each figure, the vertical axisrepresents brightness, and the horizontal axis represents a wavelength.These detection setting areas 307 and 308 are each configured as anoperation area for accepting the exposure time and the input of the gainin a predetermined unit of the pixel of the image sensor 121.

In FIGS. 13 and 14, an index 83 indicates that the excitation lightsources (L1, L2, and L4) are on, and that a longer length of theindicator 83 indicates a greater power of the light source. A detectionwavelength range of the fluorescence spectrum 85 is set by an operationbar 84. A display method of the fluorescence spectrum 85 is notparticularly limited, for example, is displayed in a total pixel averagespectrum (wavelength x intensity) at the excitation lines 1 and 2.

As shown in FIG. 13 and FIG. 14, the fluorescence spectrum 85 may bedisplayed by a heat map method in which frequency information of valuesis expressed in shading. In this case, it is also possible to visualizea signal distribution that is not clear by an average value.

Note that the vertical axis of the graph used to display thefluorescence spectrum 85 is not limited to a linear axis, and may be alogarithmic axis or a hybrid axis (biexponential axis).

It is possible to set the fluorescence spectrum 85 depending on thewavelength and the power of the excitation light source. The wavelengthrange of the fluorescence spectrum 85 can be arbitrarily changed by acursor movement operation on the operation bar 84 using an input devicesuch as a mouse. The fluorescence spectrum 85 is represented by acurrent average or a waveform calculated from the last captured waveformin consideration of a setting change.

The control unit 2 sets a read area of the image sensor 121 on the basisof the wavelength band input to the detection setting areas 307 and 308(set value). On the basis of the wavelength band set by the detectionsetting areas 307 and 308 and a predetermined conversion formula(conversion formula to a pixel corresponding to wavelength) obtained inadvance, a sensor coordinate is specified and the exposure time and thegain are set. A display area capable of individually setting theexposure time and the gain may be provided separately. The detectionsetting areas 307 and 308 display the fluorescence spectrum 85 aftersetting on the basis of the exposure time and the gain input and set viathe operation bar 84.

FIG. 15 shows an example of a screen configuration of the spectrumautomatic setting control area 309. In the spectrum automatic settingcontrol area 309, an automatic setting key 86, a histogram window 87,and the like are arranged. The automatic setting start key 86automatically performs pre-sampling imaging and the above-describedspectrum detection setting. The histogram window 87 calculates anddisplays a histogram corresponding to the wavelength range of thespectrum set in the detection setting areas 307 and 308. The verticalaxis of the histogram is frequency and the horizontal axis is thewavelength.

Referring to the histogram window 87, it is possible to explicitlyconfirm an occurrence of saturation and the presence or absence of asignal shortage (insufficient strength) when captured under thedetection conditions of the spectrum set in the detection setting areas307 and 308. In addition, it is possible to change the exposure time andthe gain while checking the histogram.

FIG. 16 is a block diagram of the fluorescence observation apparatus 100for explaining the processing executed in the control unit 2.

The control unit 2 stores the parameters set in the various settingareas 306 to 308 of the display section 3 in the storage section 21 (seeFIG. 4), and sets the read area (wavelength band), the exposure time,and the gain based on the parameter to the image sensor 121 (S401).

The control unit 2 outputs the emission spectrum of the sample Sobtained by the image sensor 121 to the display section 3 (S402), andthe waveform of the spectrum is displayed in the detection setting areas307 and 308 (see FIGS. 13 and 14).

In the automatic setting control mode, the control unit 2 executesoptimization processing of the exposure time and the gain on the basisof the captured data of the image sensor 121 (step 403), and repeatsprocessing of acquiring the captured data for the parameter afterchange.

On the other hand, when the component separation calculation of thecaptured spectrum is performed, the above-described component separationcalculation is performed on the basis of the captured data of the imagesensor 121, and the result is displayed on the display section 3 (forexample, main screen 301) (S404).

As described above, according to the present embodiment, on the basis ofthe set wavelength band, the exposure time, and the gain, the spectrumand the histogram after setting are captured and displayed in real time,and the spectrum and the histogram at a new set value are displayed fromthe obtained spectrum. Thus, the relationship between the settingparameter and the capture range, the relationship between the settingparameter and the sensor output, etc. can be comprehensively displayedand set.

Modifications

Next, a modification of the configuration of the fluorescenceobservation apparatus 100 described above will be described.

FIG. 17 is a schematic block diagram of a fluorescence observationapparatus 200 according to Modification 1, and FIG. 18 is a schematicblock diagram of a fluorescence observation apparatus 300 according toModification 2. The fluorescence observation apparatuses 200 and 300each includes the apparatus main body 1, the control unit 2, the displaysection 3, and a control program 81.

The control program 81 is a program for causing the fluorescenceobservation apparatuses 200 and 300 to execute the same function as thecontrol function performed by the control section 80 of the fluorescenceobservation apparatus 100 described above. In the fluorescenceobservation apparatus 200 shown in FIG. 17, the control program 81 isprovided in a state of being stored in a recording medium, for example,such as a magnetic disk, an optical disk, a magneto-optical disk, or aflash memory, and is downloaded to and used by an electronic computer Cor the like connected to the fluorescence observation apparatus 200.

On the other hand, in the fluorescence observation apparatus 300 shownin FIG. 18, the control program 81 distributed from outside via anetwork such as the Internet is downloaded to the electronic computer Cor the like and used. In this case, the fluorescence observationapparatus 300 and a code used to obtain the control program 81 arepackaged to be provided.

The electronic computer C in which the control program 81 is downloadedobtains various data for controlling the excitation light source 101,the spectroscopic imaging section 102, the scanning mechanism 104, thefocus mechanism 105, the non-fluorescence observation section 106, andthe like, and a control algorithm of the downloaded control program 81is executed, and control conditions of the fluorescence observationapparatuses 200 and 300 are calculated. The electronic computer C issuesa command to the fluorescence observation apparatuses 200 and 300 on thebasis of the calculated conditions, whereby the conditions of thefluorescence observation apparatuses 200 and 300 are automaticallycontrolled.

Although the embodiments of the present technology are described above,it goes without saying that the present technology is not limited to theabove-described embodiments and various modifications can be made.

The present technology may also have the following structures.

(1) A spectroscopic imaging apparatus, including:

a spectroscopic section that disperses incident light for eachwavelength;

an image sensor configured to be capable of setting an exposure time ora gain in a unit of a pixel, the image sensor detecting light of eachwavelength dispersed in the spectroscopic section; and

a control unit configured to be capable of setting the exposure time orthe gain of the image sensor in a unit of a predetermined pixel area.

(2) The spectroscopic imaging apparatus according to (1), in which

the spectroscopic section is configured to disperse the incident lightin one axial direction for each wavelength, and

the control unit is configured to set the exposure time or the gain ofthe image sensor in a unit of a line perpendicular to the one axialdirection.

(3) The spectroscopic imaging apparatus according to (1) or (2), inwhich

the image sensor includes a pixel section and a calculation section thatcalculates a pixel value from image data output from the pixel section,and

the control unit is configured to set the gain used for calculating thepixel value in the unit of the predetermined pixel area.

(4) The spectroscopic imaging apparatus according to any one of (1) to(3), in which

the control unit includes an evaluation section that obtains an emissionspectrum of the incident light on a basis of an output of the imagesensor, and a storage section that stores a plurality of referencecomponent spectra and an autofluorescence spectrum, and

the evaluation section is configured to calculate a component ratio ofthe emission spectrum such that a linear sum of a plurality of thereference component spectra and the autoluminescence spectrum isobtained.

(5) The spectroscopic imaging apparatus according to (4), in which

the evaluation section is configured to calibrate at least one of theemission spectrum or the component spectra on a basis of the exposuretime or the gain set for each predetermined pixel area.

(6) The spectroscopic imaging apparatus according to (5), in which

the evaluation section is configured to determine whether or not thereis a pixel whose pixel value reaches saturation from the capturedspectrum, and exclude the pixel reaching the saturation from calculationof a component ratio of the captured spectrum.

(7) A fluorescence observation apparatus, including:

a stage capable of supporting a fluorescence stained pathologicalspecimen;

an excitation section that irradiates the pathological specimen on thestage with line illumination;

a spectroscopic section that disperses the fluorescence excited by theline illumination for each wavelength;

an image sensor configured to be capable of setting an exposure time ora gain in a unit of a pixel, the image sensor detecting light of eachwavelength dispersed in the spectroscopic section; and

a control unit configured to set the exposure time or the gain of theimage sensor in a unit of a predetermined pixel area.

(8) The fluorescence observation apparatus according to (7), furtherincluding:

a display section for displaying the fluorescence spectrum on a basis ofan output of the image sensor.

(9) The fluorescence observation apparatus according to (8), in which

the display section has an operation area for receiving an input of anexposure time or a gain in the unit of the predetermined pixel area.

(10) The fluorescence observation apparatus according to (8) or (9), inwhich

the display section has a display area for displaying a spectrum and ahistogram after setting on a basis of the exposure time or the gain set.

REFERENCE SIGNS LIST

2 control unit

3 display section

10 spectroscopic imaging apparatus

11 spectroscopic section

12 detection section

13 fluorescence excitation section

20 stage

21 storage section

22 evaluation section

30 pixel section

31 calculation section

100, 200, 300 fluorescence observation apparatus

121 image sensor

1] A spectroscopic imaging apparatus, comprising: a spectroscopicsection that disperses incident light for each wavelength; an imagesensor configured to be capable of setting an exposure time or a gain ina unit of a pixel, the image sensor detecting light of each wavelengthdispersed in the spectroscopic section; and a control unit configured tobe capable of setting the exposure time or the gain of the image sensorin a unit of a predetermined pixel area. 2] The spectroscopic imagingapparatus according to claim 1, wherein the spectroscopic section isconfigured to disperse the incident light in one axial direction foreach wavelength, and the control unit is configured to set the exposuretime or the gain of the image sensor in a unit of a line perpendicularto the one axial direction. 3] The spectroscopic imaging apparatusaccording to claim 1, wherein the image sensor includes a pixel sectionand a calculation section that calculates a pixel value from image dataoutput from the pixel section, and the control unit is configured to setthe gain used for calculating the pixel value in the unit of thepredetermined pixel area. 4] The spectroscopic imaging apparatusaccording to claim 1, wherein the control unit includes an evaluationsection that obtains an emission spectrum of the incident light on abasis of an output of the image sensor, and a storage section thatstores a plurality of reference component spectra and anautofluorescence spectrum, and the evaluation section is configured tocalculate a component ratio of the emission spectrum such that a linearsum of a plurality of the reference component spectra and theautoluminescence spectrum is obtained. 5] The spectroscopic imagingapparatus according to claim 4, wherein the evaluation section isconfigured to calibrate at least one of the emission spectrum or thecomponent spectra on a basis of the exposure time or the gain set foreach predetermined pixel area. 6] The spectroscopic imaging apparatusaccording to claim 5, wherein the evaluation section is configured todetermine whether or not there is a pixel whose pixel value reachessaturation from the captured spectrum, and exclude the pixel reachingthe saturation from calculation of a component ratio of the capturedspectrum. 7] A fluorescence observation apparatus, comprising: a stagecapable of supporting a fluorescence stained pathological specimen; anexcitation section that irradiates the pathological specimen on thestage with line illumination; a spectroscopic section that disperses thefluorescence excited by the line illumination for each wavelength; animage sensor configured to be capable of setting an exposure time or again in a unit of a pixel, the image sensor detecting light of eachwavelength dispersed in the spectroscopic section; and a control unitconfigured to set the exposure time or the gain of the image sensor in aunit of a predetermined pixel area. 8] The fluorescence observationapparatus according to claim 7, further comprising: a display sectionfor displaying the fluorescence spectrum on a basis of an output of theimage sensor. 9] The fluorescence observation apparatus according toclaim 8, wherein the display section has an operation area for receivingan input of an exposure time or a gain in the unit of the predeterminedpixel area. 10] The fluorescence observation apparatus according toclaim 8, wherein the display section has a display area for displaying aspectrum and a histogram after setting on a basis of the exposure timeor the gain set.